SPECIFICATION 



TO ALL WHOM IT MAY CONCERN : 

BE IT KNOWN THAT I, YUKI MATSUSHIMA, a 
citizen of Japan residing at Kanagawa, Japan have 
invented certain new and useful improvements in 

CAMERA APPARATUS AND METHOD OF TAKING PICTURES 

of which the following is a specification:- 
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The present invention generally relates to a 
camera apparatus , and particularly relates to a digital 
camera which attends to different image processing, 
depending on different importance of image areas. 
5 [Description of the Background Art] 

Conventionally used cameras are mostly analog 
cameras that expose films to light. Data photographed 
by such analog cameras are generally referred to as 
/i "photographs". In the art of analog cameras, various 

2t !0 schemes have been presented to attend to automatic 

control of camera functions such as focus adjustment and 
W exposure correction, which are often difficult to users 

* who are not familiar with cameras. Such automatic 

Q 

yJ control is performed, for example, by providing a 

nJ 

ft! 15 function to detect a direction of sight to identify an 

u area of the finder that the user is looking at, and by 

focusing at the point of fixation. 

Japanese Patent Laid-open Application No. 6- 
138377 discloses an analog camera, which automatically 
20 adjusts focus by use of a sight detection means for 

detecting a fixation point of the user eye and a focus 
detection means for detecting focus points at a 
plurality of positions. Japanese Patent Laid-open 
Application No. 8-46833 discloses an analog camera which 
25 attends to automatic focus control and automatic 
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exposure control by use of a sight detection means for 
detecting the fixation point of the user eye. 

Cameras that are generally referred to as a 
digital still camera (hereinafter, "digital camera") 
5 convert optical image data acquired through a solid 

camera device such as a CCD into electrical image data, 
and store the image data in memory such as a memory card 
after proper image processing and image compression. 

Data acquired by a digital camera is referred 

10 to as an image since it is represented by electrical 

signals . With recent development of digital medium and 
the Internet, there has been an increasing demand for 
digital cameras- Different from analog cameras, digital 
cameras allows photographed images to be immediately 

15 checked, which provides an opportunity to take another 
picture in case of failure, and, also, does not require 
development of films. Such features explain part of the 
reasons why a market for digital cameras has been 
expanding. For example, there are increasing numbers of 

20 users who use digital cameras instead of memo notes when 
they attend business shows. 

In the digital cameras, process steps that 
correspond to film development of analog cameras may 
include 1) image processing and 2) image compression. 

25 At the image processing step 1), color correction is 
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necessary, for example, if a single plate camera device 
of a general type having only one color per pixel is 
used. Further, various image processing will be 
performed to improve image quality by attending to 
5 aperture processing (i.e., edge enhancement or sharpness 
enhancement), exposure correction, etc. 

Even when analog cameras are used, correction 
to some extent may be made at the time of film 
^ development- Sharpness of the photograph and condition 

10 of exposure are almost uniquely determined by graininess 
Jl: of the film (i.e., film sensitivity) and focusing at the 

y time of taking pictures . 

^ Digital cameras, on the other hand, can 

Ul perform image processing properly, thereby producing an 

ry 

RJ 15 image having better image quality than an original image 

u obtained at the time of taking pictures. Further, image 

processing can improve sharpness as well as attend to 
binary processing, which facilitates use of digital 
cameras for taking pictures of texts while analog 
20 cameras are rarely used for such purpose. 

Image processing is vital in digital cameras 
as described above, and numbers of patent applications 
have been filed in the art of image processing. 
Japanese Patent Laid-open Application No. 11-110551 
25 discloses a color discriminating device, which Is used 
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to eliminate black area noise caused by a failure to 
✓ 

determine colors during color correction process in the 
digital cameras. If the area of application is not 
restricted to digital cameras, there are numerous 
5 numbers of image processing schemes in the art of image 
processing of digital images, and these schemes are 
generally applicable to digital cameras. For example, 
Japanese Patent Laid-open Application No. 10-191064 
discloses an image filtering method and apparatus that 

10 can enhance sharpness and remove noise at the same time 
by applying filtering with weight factors that reflect 
the magnitude of contrast in the vertical and horizontal 
directions around a pixel of interest. 

In conventional schemes in the image 

15 processing of digital cameras, processing is generally 
applied across the board to the entire image. In some 
schemes, however, images are divided for processing for 
the image compression processing 2) that is performed 
prior to storage of image data in memory such as a 

20 memory card. 

Japanese Patent Laid-open Application No. 10- 
229 544, for example, assumes a direct connection of a 
digital camera and a printer. Since the size of memory 
medium provided in the digital camera is limited, the 

25 image processing apparatus disclosed in this document 
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divides an image prior to compression, rather than 
compresses/decompresses the entirety of processed data 
at once . 

Further, various image processing apparatuses 
5 that takes advantage of the fact that image data is 
acquired by a camera are also known to date. 

Japanese Patent Laid-open Application No. 9- 
326025, for example, discloses an image processing 
apparatus which retrieves only the image data of an 
10 object located at a particular distance by storing range 
data that is indicative of distance between the camera 
and objects. In this scheme, each portion of the 
acquired image can be extracted according to an 
indication of distance, thereby making it possible to 
15 take into account intention of the camera user 
subsequently at the time of editing. 

This scheme, however, requires a memory area 
for storing range data. In the worst case, a memory 
space is necessary that stores range data for each pixel 
20 of the image. A typical digital camera at present has 
1.5 million pixels, and the number of pixels will 
continue to increase in the future. This scheme is thus 
not realistic. 

Moreover, neither Japanese Patent Laid-open 
25 Application No. 10-229544 nor Japanese Patent Laid-open 
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Application No. 9-326025 takes advantage of the fact 
that image data to be processed is acquired by use of a 
camera . 

[Issues to be Addressed by the Invention] 
5 Image data that is acquired by use of a camera 

should be treated in accordance with its nature. If 
general image processing is applied without any regard 
to this fact, processing time becomes increasingly 
lengthy as processing becomes increasingly sophisticated 
?y 10 in order to improve image quality. 

Jj Ideally, image quality should be as high as 

^ possible across the entire image regardless of greater 

JL or smaller importance of different image portions . If 

y»J the highest image quality is to be maintained for the 

15 entire image, however, processing time becomes lengthy. 
M This is the reason why a user using a digital camera 

often misses a chance to take a picture at an important 
moment because of a shutter lag, which is a time period 
from a moment at which a shutter button is pressed first 
20 to a moment at which the shutter button can be pressed 
again, and is also referred to as a photographing 
interval. If the shutter lag is designed to be shorter 
to enhance the performance that is expected of a camera, 
processing time set aside for image processing, image 
25 compression, and image storing needs to be shortened. 
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As described above, conventional digital still 
cameras have drawbacks in that the user is likely to 
miss a shutter chance because of a lengthy processing 
time if sophisticated processing steps are used in the 
5 image processing to improve image quality. 

In the art of analog cameras , on the other 
hand, there is a known scheme that uses a function to 
detect a direction of user sight for the purpose of 
w " controlling focus and exposure. In analog cameras, 

>i 10 however, such function to detect a direction of user 

sight has only been used for controlling of focus and 
W. exposure, and is devoid of any concept that suggests use 

g of such function for improving image quality. When the 

y| drawbacks of digital cameras regarding the lengthy image 

nj 15 processing time is taken into consideration, use of a 

J function to detect a direction of user sight in digital 

cameras will make it possible to identify areas of 
importance in the acquired image in the manner that 
reflects intension of the camera user, 
20 Accordingly, it is an object of the present 

invention to provide a camera apparatus which can reduce 
processing time by simplifying image processing in areas 
of small importance and by taking advantage of the fact 
that the image data is acquired by a camera. 
25 In detail, it is an object of the present 
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invention to provide a digital camera which can improve 
image quality in a short processing time by determining 
image quality of image areas to reflect the intension of 
a user based on the function of detecting a line of 
5 sight of the user eye and by simplifying the image 
processing in the areas of small importance, 
[Means to Achieve Objects] 

A camera apparatus according to the present 
invention includes a camera unit which acquires an image, 

10 a line~of -sight detection unit which detects a point of 
eye fixation of a user within a camera screen, an 
importance computation unit which determines levels of 
importance for respective areas of the image acquired by 
said camera unit in accordance with the detection by 

15 said line-of -sight detection unit, and a number -of -gray- 
scale-level determining unit which changes a number of 
gray scale levels for the respective areas of the image 
in response to the determination by the importance 
computation unit. 

20 In the camera apparatus as described above, 

the number of gray scale levels is changed according to 
the levels of importance corresponding to the respective 
areas of the image, and is decreased in the areas of 
small importance, thereby producing an image having 

25 varying image quality for respective areas to reflect 
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the intension of the camera user. By decreasing the 
number of gray scale levels in the areas of small 
importance, it is possible to reduce the memory size and 
to step up the processing speed of image processing that 
5 is to be performed at later stages. 

According to another aspect of the present 
invention, a camera apparatus includes a camera unit 
which acquires an image , a line-of -sight detection unit 
which detects a point of eye fixation of a user within a 

10 camera screen, an importance computation unit which 

determines levels of importance for respective areas of 
the image acquired by said camera unit in accordance 
with the detection by said line-of - sight detection unit, 
and a color interpolation processing unit which changes 

15 color interpolation processing for the respective areas 
of the image in response to the determination by the 
importance computation unit . 

In the camera apparatus as described above, 
the color interpolation processing is changed according 

20 to the levels of importance corresponding to the 

respective areas of the image, thereby producing an 
image having varying image quality for respective areas 
to reflect the intension of the camera user. By 
simplifying the color interpolation processing in the 

25 areas of small importance, it is possible to achieve 
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high- speed color interpolation processing. 

According to another aspect of the present 
invention, a camera apparatus includes a camera unit 
which acquires an image, a line -of -sight detection unit 
5 which detects a point of eye fixation of a user within a 
camera screen, an importance computation unit which 
determines levels of importance for respective areas of 
the image acquired by said camera unit in accordance 
with the detection by said line-of -sight detection unit, 

10 and a sharpness enhancement processing unit which 
changes sharpness enhancement processing for the 
respective areas of the image in response to the 
determination by the importance computation unit . 

In the camera apparatus as described above, 

15 the sharpness enhancement processing is changed 

according to the levels of importance corresponding to 
the respective areas of the image, thereby producing an 
image having varying image quality for respective areas 
to reflect the intension of the camera user. By 

20 simplifying the sharpness enhancement processing in the 
areas of small importance, it is possible to achieve 
high-speed sharpness enhancement processing. 

According to another aspect of the present 
invention, a camera apparatus includes a camera unit 

25 which acquires an image, a line-of -sight detection unit 
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which detects a point of eye fixation of a user within a 
camera screen, an importance computation unit which 
determines levels of importance for respective areas of 
the image acquired by said camera unit in accordance 
5 with the detection by said line -of -sight detection unit, 
and a noise removal processing unit which changes noise 
removal processing for the respective areas of the image 
in response to the determination by the importance 
computation unit . 

10 In the camera apparatus as described above, 

the noise removal processing is changed according to the 
levels of importance corresponding to the respective 
areas of the image, thereby producing an image having 
varying image quality for respective areas to reflect 

15 the intension of the camera user. By simplifying the 
noise removal processing in the areas of small 
importance, it is possible to achieve high-speed noise 
removal processing . 
[Embodiments of the Invention] 

20 In the following, a digital still camera 

according to the present invention will be described 
with reference to the accompanying drawings . 

Fig . 1 is a block diagram showing an embodiment 
of a digital camera according to the present invention. 

25 In Fig.l are shown a camera lens l, a solid camera 
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device (CCD) 2, an A/D conversion unit 3, a camera unit 
4 that includes the solid camera device 2 and the A/D 
conversion unit 3, an image processing unit 5, an image 
compression unit 6, an image processing/compression unit 
5 7 that includes the image processing unit 5 and the 

image compression unit 6, a memory unit 8, a PC display 
9, a line-of -sight detection unit 10, a line-of -sight 
data storage unit 11, an important area identifying unit 
12, an importance computation unit 13, an Importance 

10 storage unit 14, and an image quality determining unit 

15 that includes the image processing/compression unit 7, 
the important area identifying unit 12, the importance 
computation unit 13, and the importance storage unit 14. 

When the camera user see through the finder of 

15 the digital camera shown in Fig.l according to the 

embodiment of the present invention, the line-of - sight 
detection unit 10 detects a fixation point of the user 
eye. Data of the line of sight is temporarily stored in 
the line-of -sight data storage unit 11, and, then, the 

20 important area identifying unit 12 identifies areas of 
importance in the image. Further, the importance 
computation unit 13 computes importance of each area in 
the image, and the computed importance is stored in the 
importance storage unit 14. There are various available 

25 methods for detecting a fixation point of the user eye. 
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and any one of these methods can be used in the present 
invention. An example of the line-of -sight detection 
unit 10 will be described later. 

A CCD is only an example of the solid camera 
5 device 2, and the solid camera device 2 is not limited 
to the CCD. For the sake of description, the solid 
camera device is hereinafter described as a CCD. A CCD 
carries out photoelectric conversion of the object image 
formed by the optical camera lens 1, and outputs 

10 electrical signals. 

After this, the A/D conversion unit 3 converts 
analog output signals of the CCD solid camera device 2 
into digital signals . 

The image processing unit 5 provided at the 

15 next processing stage performs various processing to 

improve image quality with respect to the digital signal 
outputs of the A/D conversion unit 3, and outputs image 
data. A detailed description of the image processing 
unit will be provided later. 

20 The image compression unit 6 compresses the 

image data output from the image processing unit 5. The 
compressed image data is referred to as a data stream. 
Various schemes has been proposed to date with regard to 
image compression. As a compression method for 

25 compressing images acquired by digital cameras, a method 
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of a well-established image compression algorithm such 
as JPEG has been used. Notwithstanding this, an image 
compression/decompression method that has been presented 
by the applicant , for example , may be used to achieve 
5 efficient image compression while maintaining image 
quality of important areas specified in the image. 
(Japanese Patent Application No. 10-175999: An image 
processing method specifies an important area in the 
image, and computes importance of retrieved image data 

10 by using as a parameter the distance between the area of 
importance and the location of the retrieved image data, 
followed by changing a rate of compression or a rate of 
decompression of each pixel of the image data in 
response to the computed importance.) In this manner, 

15 efficient compression is achieved by use of data of 
importance stored in the importance storage unit 14 . 

The image processing unit 5 and the image 
compression unit 6 are collectively referred to as the 
image processing/compression unit 7. Further, the 

20 important area identifying unit 12, the importance 

computation unit 13, the importance storage unit 14, and 
the image processing/compression unit 7 are collectively 
called the image quality determining unit 15- 

The image quality determining unit 15 is 

2 5 provided with a function to change image quality of 
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image areas, depending on the position of the fixation 
point of the user eye. In this manner , the image 
quality determining unit 15 can provide image quality 
that reflects intension of the camera user. A detailed 
5 description of this aspect will be provided later. 

The memory unit 8 stores the data stream 
therein. As the memory unit 8, a floppy disk, an MO r a 
memory card, etc. , may be used, but the memory unit 8 is 
not limited to a particular structure. The memory unit 
10 8 may be a memory device connected to a computer system 
via a cable, so that the digital camera is connected to 
the computer as an image input device. 

The data stream stored in the memory unit 8 
can be reproduced on the display 9 of a personal 
15 computer or the like. There are other forms of 

reproduction that are conceivable such as a printer 
output, and the form of reproduction is not limited to 
any particular embodiment. 

In the following, a configuration of each unit 
20 relevant to the present invention in Fig.l will be 
described together with details of operation. 

Fig . 2 is a drawing showing an embodiment of 
the line-of -sight detection unit 10. 

The line-of -sight detection unit 10 includes 
25 an ocular 20, a dynamic mirror 21, an infrared light 



-22- 



emission diode 23, a condenser 24 , a photoelectric- 
conversion device 25, a fixation point detection circuit 
26, and a video display device 27. 

The dynamic mirror 21 allows passage of 
5 visible light and reflects infrared light, and is 

situated in front of the optical ocular 20. In front of 
the ocular 20, also, the infrared light emission diode 
23 is provided to emit infrared light to an eye 22 of 
the camera user. The infrared beam emitted from the 

10 infrared light emission diode 23 is reflected by the eye 
22 of the camera user, and is further reflected by the 
dynamic mirror 21 to reach the photoelectric-conversion 
device 25 after passing through the condenser 24. The 
condenser 24 is provided in such a position a.s to form 

15 an image of the iris of the eye 22 on the conversion 
plate of the photoelectric -conversion device 25. 

The configuration described above forms an 
image on the conversion plate of the photoelectric- 
conversion device 25 where this image is an image of 

20 beam spot of the infrared light emission diode 23 
superimposed on the eye 22 of the camera user. 

The fixation point detection circuit 26 
detects the position of the fixation point on the 
display screen of the video display device 27 based on 

25 relation between the position of the infrared beam spot 
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and the position of the iris on the photoelectric 
conversion plate of the photoelectric-conversion device 
25. 

Here, the display screen may be divided into a 
5 plurality of blocks, and the fixation point may be 
detected by the unit of one block. 

In what follows, the operation of the 
important area identifying unit 12 and the importance 
computation unit 13 will be described in detail. 

10 As the simplest example, a case in which an 

area of importance is provided as a single point in the 
image will be described first. Namely, the area of 
importance is a single point that corresponds to the 
fixation point of the camera user detected by the line- 

15 of -sight detection unit 10. In the case where the 

fixation point is detected by the unit of one block, the 
area of importance is regarded as a center point of the 
block. 

The important area identifying unit 12 
20 identifies a single point as an "area of importance" 

based on the position of the fixation point of the user 
eye. In response, the importance computation unit 13 
uses a formulae for computing importance based on a 
parameter indicative of a distance between the area of 
25 importance and any given point in the image data, 
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thereby computing importance for each position in the 
image. The computation formulae for computing 
importance in a continuous scale may be represented as: 
Importance = a X + b ( 1 ) 

5 a, b: real values, 

X: distance from the area of importance 
Fig. 4 is a drawing showing importance of a 
pixel of interest and positional relationship between 
the pixel of interest and the area of importance. 

10 As shown in Fig. 4, when a single point is 

given as the area of importance, the importance of the 
point of interest is computed as a linear function of 
the distance X between the point of interest and the 
area of importance. 

!5 In what follows, a description will be given 

with regard to a case in which the area of importance is 
provided as a small circular area in the image. 

In such a case, a function that is applied to 
the area of interest has a distribution based on 

20 parameters that include the distance from the center of 
the small area, the size of the small area, and the 
magnitude at the center of the small area. Based on 
this distribution function, the importance of any given 
point is computed with respect to the image data. 

25 The important area identifying unit 12 
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identifies the area of importance with, respect to the 
object image by using the position of the fixation point 
of the user eye. In this example, the area of 
importance is defined as a small circular area around 
5 the point of fixation detected by the line-of -sight 
detection unit 10, This is done by determining the 
radius and center point of the circle that defines the 
area of importance. 

The importance of each pixel can be 

10 represented by using a Gaussian function f (x) . A 

Gaussian function is represented by the formulae as 
shown below as equation (2). In this representation, 0 
is a standard deviation of the distribution, and u is 
an average, with x representing the distance from the 

15 average. Further, K Is a constant for normalization. 

f(x) = {K/(27C a 2 ) 1/2 }[exp{-(x-/X ) 2 /20 2 }] (2) 
Fig. 5 is a drawing showing the Gaussian 
distribution function f (x) that defines importance. 

In the equation (2), the center point of the 

20 area of importance (the fixation point of the user eye) 
is provided as U (=0), and the distance from the center 
point is represented by x. In this manner, importance 
at any given point is computed as f (x) of the equation 
(2). Further, o is controlled in proportion to the 

25 radius of the area of interest to adjust the size of the 
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distribution of the distribution function f (x) , thereby 
providing proper importance commensurate with the size 
of the area of importance. 

Moreover, the magnitude of the greatest 
5 importance that is provided at the center of the area of 
importance, i.e., at the point of fixation of the user 
eye, is set equal to a proper value by use of the 
normalization factor K. 

Namely, importance computed by the importance 
10 computation unit 13 based on the distance x from the 
center point of the area of importance is represented 
as : 

f(x) - {K/{2rc a 2 ) 1/2 }[exp{-x 2 /2a 2 }] (3) 
It should be noted that the distribution 
15 function is not limited to a Gaussian function. There 
are various functions having a distribution that 
reflects the radius of the area of importance, and any 
one of such functions may be used. 

In the following, a description will be 
20 provided with respect to a case in which there are a 

plurality of areas of importance. That is, an area of 
importance is a point of fixation of the user eye, and 
such area of importance is provided at multiple 
locations . 

2 5 Fig. 6 is a drawing showing a case in which two 



areas of importance are provided in an image. 

A first area of importance (area 1) is located 
at a distance rl from the pixel of interest (x, y), and 
a second area of importance (area 2) is located at a 
distance r2 from the pixel of interest (x F y) . 

Fig -7 is a drawing showing the importance of 
the position (x, y) by representing the distance between 
the position (x, y) and the first area of importance and 
the distance between the position (x, y) and the second 
area of importance in the horizontal axis. 

As shown in Fig. 7, the importance computation 
unit 13 computes the importance a(x. y) at the position 
(x, y) attributable to the second area of importance, 
and computes the importance & (x, y) at the position (x, 
y) attributable to the first area of importance. Total 
importance P(x, y) at the position (x, y) is represented 
as a sum of the importance values attributable to all 
the areas of importance. 

P(x, y) = (a(x, y) + /3 (x, y)) 

The same computation applies in the case of 
more than two areas of importance. 

In the example of the equation (3), the total 
importance is obtained as a sum of all the importance 
values. This example is not intended to be a limitation, 
and, for example, a product may be used in place of a 
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sum. 

In what follows, a description will be given 
with respect to a case in which an area of importance is 
defined as a small ellipse region of the image. 

In such a case, two importance distribution 
functions orthogonal to each other and having different 
characteristics are employed to form a distribution 
function that has an ellipse-shape distribution. 

For example, two Gaussian functions fx(x) and 
fy(y) orthogonal to each other may be used to form a 
distribution function F(x, y) . Fig. 8 is a drawing 
showing contours of importance obtained by F(x, y) . 

Although examples of computation of continuous 
importance have been given in the description provided 
above, importance may assume discreet values. 

Fig. 9 is a drawing showing an example in which 
the importance level is quantized into 5 levels . 

The importance defined in this manner is 
stored in the importance storage unit 14. In the 
description of embodiments provided below, quantized 
importance will be used. 

In the following, a configuration and 
operation of the image processing unit 5 will be 
described. 

Fig. 10 is a block diagram showing a functional 
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block configuration of the image processing unit 5. 

As a first step, a functional block 5-1 
retrieves importance A, B, and C(A>B>C)of 
respective areas from the importance storage unit 14. 
5 Fig* 11 is a drawing showing quantized 

importance assigned to respective areas in the image in 
an example in which an area of importance has an ellipse 
shape. As shown in Fig. 11, an image area (area 1) 
closest to the fixation point of the user eye has 

10 importance A assigned thereto, and the importance A is 
the greatest importance. Further, an area (area 2) 
distanced from the fixation point and surrounding the 
area of the importance A has importance B assigned 
thereto. Moreover , the remaining image area (area 3) 

15 has the smallest importance C assigned thereto. 

A functional block 5-2 receives image data 
output from the A/D conversion unit 3 shown in Fig.l. 

A functional block 5-3 matches the importance 
values of the respective areas retrieved from the 

20 importance storage unit 14 with the image data output 
from the A/D conversion unit 3, thereby allocating 
importance to each area of the image data. The greater 
the importance , the more important the area is , so that 
a functional block 5-4 determines image quality levels 

2 5 in accordance with importance of the respective areas 1 
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through 3 . 

After this, a functional block 5-5 performs 
image processing so as to provide image quality that is 
the highest for the importance A, intermediate for the 
5 importance B, and the lowest for the importance C. For 
example, the image data is subjected to processing for 
determining the number of gray scale levels, processing 
for color interpolation, processing for improving 
sharpness, processing for removing noise, etc. Each of 

10 these processing will be described later in detail in 
conjunction with a description of embodiments. 

It should be noted that the determination of 
importance may be made by the unit of one block rather 
than by the unit of one pixel where an image is divided 

15 into a plurality of blocks. 

Fig. 12 is an example in which an image is 
divided into a plurality of blocks arranged in rows and 
columns. In this example, the image is divided into a 
plurality of blocks arranged in rows and columns, and 

20 importance is determined separately with respect to each 
block, thereby attending to processing that provides 
varying image quality. In this case, processing can be 
simplified compared to the case in which importance is 
determined with respect to each pixel to attend to 

25 different processing for each pixel. 
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Fig. 13 is an example in which an image is 
divided into a plurality of blocks that are not the same 
size and shape. In this example, a consideration is 
given to the fact that an image is acquired by a digital 
5 camera , so that size of a block is changed between the 
center and the periphery. Division into blocks is not 
limited to specific configurations provided above, and a 
proper method of dividing an image may be employed by 
taking into consideration application fields and design 
10 conditions . 

In the following, an embodiment of image 
output apparatus will be described - 

Fig. 14 is a block diagram showing an example 
of a hardware configuration including image output 
15 apparatus . 

In the system of Fig. 14, a digital camera 201 
is used as an image input device. The digital camera 
201 is connected to a computer system that includes a 
computer 210, a hard drive 203, a keyboard 204, a CD-ROM 

20 drive 205, a floppy disk drive 206, and a modem 207. 

The image output apparatus operates through the computer 
210. An example of the image output apparatus includes 
a printer 209 such as an ink- jet printer, a display 208, 
a floppy disk 216, an MO 217, and a server 215 for 

25 storing data therein. 
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The modem 207 is connected to an external 
network via a public communication line, and can 
download software and data to the computer system. 

An operating system 212 runs on the computer 
5 210, which has a printer driver 214 and a display driver 
213 installed therein, corresponding to the printer and 
the display. When using the image output apparatus, the 
computer 210 receives RGB image-level data, and has the 
image displayed on the display 208 via the display 
Jri 10 driver 213, for example. The computer 210 may have the 

Hf image data stored in the server 215, or may use the 

W printer driver 214 to convert the image data into CMY or 

5 CMYK binary data, which is then printed by the printer 

LJ 209. The computer 210 may store the data in the floppy 

fy 15 disk 216 or the MO 217 serving as a memory medium. 

\2 In the following, an embodiment of processing 

that makes image quality different for different levels 
of importance will be described in connection with the 
digital camera of the present invention. This 
20 processing is performed by the image processing unit 5. 

Fig. 15 is a flowchart of a first embodiment of 
processing that makes image quality vary depending on 
levels of importance . 

In the first embodiment, a nature of gray- 
25 scale representation is given a greater^ consideration in 
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areas of great importance than in other areas. This 
makes it possible to improve image quality in the areas 
of great importance, and, at the same time, can reduce 
the required memory volume and step up the image 
processing speed compared with a case in which the same 
level of gray- scale representation is applied across the 
board to the entire image. 

At a step ST1, importance of each area is 
retrieved from the importance storage unit 14. At a 
step ST2 , image data is supplied from the A/D conversion 
unit 3. 

At a step ST3 , importance is determined by 
identifying an area to which the pixel X of interest 
belongs- At a step ST4, the means for determining the 
number of gray scale levels determines the number of 
gray scale levels . When the areas 1 through 3 have 
importance levels A through C (A > B > C) as in the 
example of Fig. 11, for example, the means for 
determining the number of gray scale levels changes the 
number of gray scale levels such as to achieve the 
following relation. 

Number of gray scale levels for area 1 

> Number of gray scale levels for area 2 

> Number of gray scale levels for area 3 . 
Examples of the numbers of gray scale levels may be: 
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Number of gray scale levels for area 1 = 256; 
Number of gray scale levels for area 2 = 128; and 
Number of gray scale levels for area 3 = 64. 
In the case where the digital camera is 
5 directly connected to a printer or the like to output 

the image, for example, the numbers of gray scale levels 
may be determined for respective areas in accordance 
with the number of output gray scale levels of the 
printer. If the number of gray scale levels of the 
10 printer is 128, for example, the numbers of gray scale 
levels may be: 

Number of gray scale levels for area 1 = 128; 
Number of gray scale levels for area 2 = 64; and 
Number of gray scale levels for area 3 - 32. 
15 The numbers of gray scale levels are typically 

determined in accordance with performance of the output 
apparatus or the like, and may vary depending on the 
apparatus that is actually used. 

At a step ST5 , a check is made as to whether 
20 the process for determining the number of gray scale 

levels is completed for all the pixels. If the process 
for determining the number of gray scale levels is not 
completed for all the pixels, the procedure goes back to 
the step ST3, and processing of a next pixel is carried 
25 out. If the process for determining the number of gray 
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scale levels is completed for all the pixels, the 
procedure goes to a step ST6 . 

At a step ST6 , the image is output - This 
marks the end of the procedure . 
5 According to the first embodiment as described 

above, the number of gray scale levels is changed 
depending on the importance of respective areas . The 
smaller the importance of an area, the smaller the 
number of gray scale levels for this area is, thereby 

10 producing an image that has varying image quality for 

respective areas to reflect the intension of the camera 
user. By decreasing the number of gray scale levels 
with a decrease in importance, it is possible to reduce 
the required memory size and to achieve high-speed image 

15 processing that is to be performed at subsequent stages. 
Fig. 16 is a flowchart of a second embodiment 
of processing that makes image quality vary depending on 
levels of importance . 

When a camera device is a CCD with a primary 

20 color array, a color filter as shown in Fig. 17 is used. 
In this case, any given pixel is provided with only one 
color selected from R, G, and B, so that colors of the 
two remaining colors need to be interpolated. A process 
for interpolating color information is referred to as 

25 color interpolation processing. 
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As can be seen from Fig. 17, the density of G 
pixels is twice as great as the density of R pixels or 
the density of B pixels in the primary color array. 
This is because human visual perception of luminance is 
5 relatively greater for the G component. Interpolation 
of the G color is thus performed more rigidly than 
interpolation of the two other colors . 

In this embodiment, color interpolation 
processing is given a greater consideration in areas of 

10 great importance than in other areas . This makes it 

possible to improve image quality in the areas of great 
importance, and, at the same time, can step up the 
color-interpolation processing speed compared with a 
case in which the same image processing is applied 

15 across the board to the entire image. 

In Fig. 18, a pixel X located at the center of 
the figure is the pixel of interest. Analysis will be 
given below in respect of a case in which the G-color 
information of the pixel X is interpolated. The four 

20 neighboring pixels of the pixel X are referred to as 
pixels a, b, c, and d, which have G values that are 
already known. The G value of the pixel X is to be 
computed based on the G values of the pixels a, b, c, 
and d . 

25 An absolute value of a difference between the 
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pixel b and the pixel c is denoted as Ax, and an 
absolute value of a difference between the pixel a and 
the pixel d is denoted as Ay. Ax and Ay are compared 
with each other to check whether the green color at the 
5 pixel X has a directionality either in the horizontal 
direction or in the vertical direction. Interpolation 
is applied in the direction that has a less color change. 

Namely, when either Ax or Ay is greater than 
a threshold value th, an average of the two pixels is 

10 computed in respect of the smaller of the Ax and Ay, 
thereby attending to interpolation. Otherwise, it is 
ascertained that there is no directionality either in 
the horizontal direction or in the vertical direction, 
and an average of the four pixels a, b, c, and d is 

15 obtained. Namely, 
Ax - |b - c| 

Ay - |a - d| (4) 

and 

X = (b + c)/2 if (Ax < Ay and Ay ^ th) 
20 (a + d)/2 if (Ax > Ay and Ax ^ th) (5) 

(a+d+b+c)/4 other (6) 
This is the color interpolation processing of the areas 
of great importance. 

In areas other than the areas of great 
25 importance, simple averaging computation based on the 
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equation (6) may be employed. Further, the simple 
averaging of neighboring pixels is computed in the case 
of X being R or B . In this manner, it is possible to 
attend to color interpolation processing according to 
5 importance levels, i.e., to place a greater emphasis on 
the areas of great importance than on other areas in 
terms of color interpolation processing. 

With reference to Fig. 16, at a step ST1 , 
importance of each area is retrieved from the importance 
10 storage unit 14. At a step ST2 , image data is supplied 
from the A/D conversion unit 3 . 

At a step ST3, importance is determined by 
identifying an area to which the pixel X of interest 
belongs . 

15 At a step ST4, a color to be interpolated for 

the pixel X of interest is determined. If the color to 
be interpolated is the G color, the procedure goes to a 
step ST5. If the color to be interpolated is R or G, 
the procedure goes to a step ST6 . 

20 At the step ST5, the color interpolation 

processing as described above is carried out in 
accordance with the importance of the area. At the step 
ST6, a simple average of the four neighboring pixels is 
obtained. 

25 At a step ST7, a check is made as to whether 
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the color interpolation processing is completed for all 
the pixels . If the color interpolation processing is 
not completed for all the pixels, the procedure goes 
back to the step ST3 / and processing of a next pixel is 
5 carried out. If the color interpolation processing is 
completed for all the pixels, the procedure goes to a 
step ST8. 

At the step ST8 , the image is output. This 
marks the end of the procedure. 

10 According to the second embodiment as 

described above, the color interpolation processing is 
changed depending on the importance of respective areas , 
thereby producing an image that has varying image 
quality for respective areas to reflect the intension of 

15 the camera user. By simplifying the color interpolation 
processing in the areas of less importance, it is 
possible to achieve high-speed color interpolation 
processing . 

Further, as a variation of the second 

20 embodiment, the color interpolation processing may be 
changed in accordance with importance by changing a 
range of an interpolation reference area in accordance 
of importance of respective areas . 

For example, a relatively wide area of 

25 interpolation reference is used in an area of relatively 
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great importance, and a relatively narrow area of 
interpolation reference is used in an area of relatively 
small importance. A color value of the pixel of 
interest can be determined from all the color 
5 information contained in the assigned area of 
interpolation reference . 



relatively wide area of interpolation reference that is 
employed in the case of relatively great importance. 



with respect to the area of interpolation reference 
shown in Fig. 19. Twelve neighboring pixels of the pixel 
X of interest are denoted as a, b, c, d, e, f , g, h, i, 
j, k, and 1, all of which are used to interpolate the G 
15 information of the pixel X of interest. The twelve 

neighboring pixels have G values that are already known. 



pixel of interest has a directionality either in the 
horizontal direction or the vertical direction. Namely, 
20 the color interpolation processing is performed by 
computing 



Fig. 19 is a drawing showing an example of a 



10 



A method of interpolation will be described 



Further, a check is made as to whether the 



Ax = |(i + b + j)-(l + c + k) | 
Ay = |(e + a + h) - (f + d + g) | 



(7) 



and 



25 



X = (i+b+j+l+c+ k)/6 
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if (Ax < Ay and Ay ^ th) 
(e + a + h + f + d + g)/6 

if (Ax > Ay and Ax S th) (8) 
(a+d+b+c)/4 other (9) 
5 In areas of relatively small importance , a simple 

average of four pixels a, b, c, and d may be obtained 
based on the equation (9). In this case, the area of 
interpolation reference is the same as the area shown in 
Fig. 18, which is narrower than the area of interpolation 
10 reference shown in Fig. 19. 

In the variation of the second embodiment, a 
relatively wide area of interpolation reference is used 
in an area of relatively great importance, and a 
relatively narrow area of interpolation reference is 
15 used in an area of relatively small importance, so that 
the color interpolation processing is changed depending 
on the importance of respective areas, thereby producing 
an image that has varying image quality for respective 
areas to reflect the intension of the camera user. By 
20 simplifying the color interpolation processing in the 
areas of less importance, it is possible to achieve 
high-speed color interpolation processing. 

Fig. 20 is a flowchart of a third embodiment of 
processing that makes image quality vary depending on 
25 levels of importance. 
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In the third embodiment , processing for 
improving image sharpness is changed according to the 
importance levels of respective image areas. The 
processing for improving sharpness is also called an 
5 aperture correction, which enhances edge portions of the 
image. In order to enhance edge portions, a check needs 
to be made first as to whether the pixel of interest is 
part of an edge portion . Such a check is made by using 
G signals based on the assumption that G values are good 

10 representative of luminance levels. 

In this embodiment, the check for determining 
whether the pixel of interest is part of an edge portion 
and the processing for improving sharpness are performed 
more diligently in areas of great importance than in 

15 other areas. This makes it possible to improve image 
quality in the areas of great importance, and, at the 
same time, can step up the processing speed compared 
with a case in which the same image processing is 
applied across the board to the entire image. 

20 As shown in Fig. 21, the pixel value of the 

pixel of interest is denoted as X in the area of great 
importance. With respect to the pixel X, four 
neighboring pixels in the horizontal direction have 
pixel values a, b, c, and d, and four neighboring pixels 

25 in the vertical direction have pixel values s # t, u, and 
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v. In order to detect edges, high- range Laplacian 
operation and middle-range Laplacian operation are 
performed in respect of the horizontal direction as well 
as the vertical direction, for example* 
5 High-range Laplacian in Horizontal Direction: 

Lhh = 2X - b - c 
Middle-range Laplacian in Horizontal Direction: 

Lhm = 2X - a - d 
High-range Laplacian in Vertical Direction: 
2 10 Lvh =* 2X - t - u 

Si 

pJ Middle-range Laplacian in Vertical Direction: 

Hi Lvm = 2X - s - v (10) 

1: In the equation (10), values of two neighboring pixels 

positioned above and below the pixel of interest or 
UJ 15 positioned to the left and to the right of the pixel of 

[U interest are subtracted from twice the pixel value of 

M= the pixel of interest. Namely, when the value of the 

pixel of interest is different from the values of the 
neighboring pixels, the computed value is relatively 
20 large. On the other hand, when the value of the pixel 
of interest is comparable to the values of the 
neighboring pixels, the computed value is relatively 
small. Laplacian operation as described above can 
detect the degree to which the pixel of interest has an 
25 isolated pixel value. 
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Further, Lhh, Lhm, Lvh, and Lvm are subjected 
to non-linear transformation f(x) as shown in Fig. 22. 
This puts a lower limit th_min and an upper limit th_max 
to the results of Laplacian operation. In this limiting 
5 process, if the value of the Laplacian operation is 
smaller than th_min, the value is set equal to zero, 
thereby treating the pixel of interest as being not part 
of an edge portion. 

Further, a comparison is made between (f (Lhn) 
10 + f(Lhm)) and (f(Lvh) + f (Lvm) ) to check whether the 

pixel X is part of a horizontal line or a vertical line. 
If (f(Lhh) + f(Lhm)) > (f(Lvh) + f(Lvm)), the pixel of 
interest is regarded as part of a vertical line. If 
(f(Lhh) + f(Lhm)) < (f(Lvh) + f(Lvm)), the pixel of 
15 interest is regarded as part of a horizontal line. 

If the pixel of interest is regarded as part 
of a horizontal line, the R value Xr, the B value Xb, 
and the G value Xg of the pixel X are multiplied by 
correction factors for improving the sharpness of 
20 horizontal lines. In detail, a R value Xr* , a B value 
Xb 1 , and a G value Xg' of the pixel X after the 
correction are computed as follows . 

Xr 1 = Xr((f(Lhh) + f(Lhmj)/2 + Z)/Z 
Xg' = Xg((f(Llih) + f(Lhm))/2 + Z)/Z 
25 Xb 1 = Xb((f(Lhh) + f(Lhm))/2 + Z)/Z (11) 
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It should be noted that the greater the possibility of 
the pixel of interest being part of a horizontal line, 
the larger the value of |f(Lhh) + f(Lhm)|. According to 
the equation (11), therefore, the greater the 
5 possibility of the pixel of interest being part of a 

horizontal line, the more enhanced the pixel of interest. 
According to experiments conducted by the inventor, Z is 
preferably close to 128. 

Processing for a vertical line is performed in 
10 the similar manner. Namely, if the pixel of interest is 
regarded as part of a vertical line, the R value Xr, the 
B value Xb, and the G value Xg of the pixel X are 
multiplied by correction factors for improving the 
sharpness of vertical lines . In detail , a R value Xr ' , 
15 a B value Xb ' , and a G value Xg f of the pixel X after 
the correction are computed as follows. 

Xr' = Xr((f(Lvh) + f(Lvm))/2 + Z)/Z 
Xg 1 = Xg((f(Lvh) + f(Lvm))/2 + z)/Z 

Xb' = Xb((f(Lvh) + f(Lvm))/2 + Z)/Z (12) 
20 According to the equation (12), the greater the 

possibility of the pixel of interest being part of a 

vertical line, the more enhanced the pixel of interest. 

According to experiments conducted by the inventor, Z is 

preferably close to 128. 
25 The color interpolation processing for the 
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areas of great importance is described above. 

In areas other than the areas of great 
importance, only either one of the high -range Laplacian 
operation and the middle-range Laplacian operation may 
5 be computed to check the horizontal and vertical 

components of the pixel of interest. In this manner, it 
is possible to conduct sharpness enhancement in 
accordance with the levels of importance, i.e., to place 
a greater emphasis on the areas of great importance than 
10 on other areas in terms of sharpness enhancement. 

With reference to Fig. 20, the processing 
described above will be summarized. At a step ST1, 
importance of each area is retrieved from the importance 
storage unit 14. At a step ST2 , image data is supplied 
15 from the A/D conversion unit 3. 

At a step ST3, importance is determined by 
identifying an area to which the pixel X of interest 
belongs . 

At a step ST4 , a check is made in accordance 
20 with the importance of the pixel X of interest as to 

whether the pixel of interest is part of an edge portion 
and the processing for improving sharpness is performed 
according to the levels of importance. 

At a step ST5, a check is made as to whether 
25 the sharpness enhancement processing is completed for 
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all the pixels. If the sharpness enhancement processing 
is not completed for all the pixels, the procedure goes 
back to the step ST3 , and processing of a next pixel is 
carried out. If the sharpness enhancement processing is 
5 completed for all the pixels, the procedure goes to a 
step ST 6 . 

At the step ST6 , the image is output. This 
marks the end of the procedure. 

According to the third embodiment as described 

10 above, the sharpness enhancement processing is changed 

depending on the importance of respective areas, thereby 
producing an image that has varying image quality for 
respective areas to reflect the intension of the camera 
user. By simplifying the sharpness enhancement 

15 processing in the areas of less importance, it is 

possible to achieve high-speed sharpness enhancement 
processing. 

Fig. 2 3 is a flowchart of a fourth embodiment 
of processing that makes image quality vary depending on 
2 0 levels of importance. 

Digital cameras use an electrical shutter, so 
that obtained images may sustain noise. In order to 
remove the noise, output signals of the CCD are 
subjected to optical low-pass filtering, which is 
25 incomplete, however, in that all the noise is not 
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removed. Further, color interpolation processing and 
sharpness enhancement processing may end up emphasizing 
the noise, in which case the processing of noise removal 
is absolutely necessary. 
5 Noises that are generated by color 

interpolation processing appear near boundaries between 
the black and the white. It is more likely to have 
erroneous colors in monochrome areas when R and/ or B 
components having sparse density are interpolated than 
J3 10 when G components having greater density are 

ry interpolated. Such noise is referred to as a pseudo 

7n color. 

^ In this embodiment, a process of determining 

L. whether the pixel of interest sustains a pseudo color is 

15 given a greater consideration in areas of great 

Of importance than in other areas . This makes it possible 

fill 

H= to improve image quality in the areas of great 

importance, and, at the same time, can step up the 
processing speed compared with a case in which the same 
20 image processing is applied across the board to the 
entire image. 

As a first step, a check is made as to whether 
the pixel of interest is a noise. In order to extract 
the noise of R and B components, a RB component of the 
25 pixel X of interest is computed. With the R value Xr 
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and B value Xb of the pixel X, the RB component of the 
pixel X is computed as : 

Ax_RB = |Xr - Xb| (13) 
Then, the four neighboring pixels around the 
pixel of interest are denoted a, b, c, and d as shown in 
Fig. 18, and a weighted average Ae is obtained as 
follows . 

Ae = 8 x Ax_rb 
- 2 x (Aa_RB + Ab_RB + Ac_RB + Ad_RB) (14) 
Ae is compared with a threshold value th_e. If Ae is 
greater than the threshold value th__e, it is ascertained 
that the pixel of interest is a noise. If the pixel of 
interest is a noise, noise removal is performed. In 
detail, a R value Xr 1 and a B value Xb* of the pixel X 
after the correction are computed as follows. 

Xr' = (ar + br + cr + dr)/4 if (AE S th_e) 

Xr' = Xr f other 



Xb f = (ab + bb + cb + db)/4 if ( Ae ^ th_e) 
Xb' = Xb' other 

(15) 

where ar, br, cr, and dr are R components of the pixels 
a, b f c, and d, respectively, and ab, bb, cb, and db are 
B components of the pixels a, b, c, and d, respectively. 
This is the color interpolation processing for areas of 
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great importance . 

In areas other than the areas of great 
importance, the weighted average Ae is obtained by 
using two neighboring pixels located on one side of the 
pixel of interest rather than using all the four 
neighboring pixels. Namely, it may be obtained as: 

AE - 4 x Ax_RB - 2 x (AaJRB + Ab_RB) (16) 
In this case, the noise removal is also simplified by 
using only the two neighboring pixels provided on one 
side of the pixel of interest. For example, 

Xr' = (ar + for)/2 if (Ae ^ th_e) 

Xr ' = Xr f other 



Xb 1 « (ab + bb)/2 if (AE i th_e) 

Xb' = Xb' other 

(17) 

With reference to Fig. 23, the processing 
described above will be summarized. At a step ST1 , 
importance of each area is retrieved from the importance 
storage unit 14. At a step ST2 . image data is supplied 
from the A/D conversion unit 3. 

At a step ST3, importance is determined by 
identifying an area to which the pixel X of interest 
belongs - 

At a step ST4 , a check is made in accordance 
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wlth the importance of the pixel X of interest as to 
whether the pixel of interest is a noise, and the 
processing of noise removal is performed according to 
the levels of importance. 
5 At a step STB, a check is made as to whether 

the noise removal processing is completed for all the 
pixels . If the noise removal processing is not 
completed for all the pixels, the procedure goes back to 
the step ST3, and processing of a next pixel is carried 

10 out. If the noise removal processing is completed for 
all the pixels, the procedure goes to a step ST6* 

At the step ST6 , the image is output . This 
marks the end of the procedure* 

According to the fourth embodiment as 

15 described above , the noise removal processing is changed 
depending on the importance of respective areas, thereby- 
producing an image that has varying image quality for 
respective areas to reflect the intension of the camera 
user. By simplifying the noise removal processing in 

20 the areas of less importance, it is possible to achieve 
high-speed noise removal processing. 

The noise removal processing is not limited to 
the example of the above embodiment . For example , a 
check can be made as to whether the pixel of interest is 

25 a noise by checking whether the RGB values of the pixel 
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of interest fall within a predetermined range around the 
RGB averages of the reference area. Namely, two 
positive threshold values may be provided as an upper 
limit and a lower limit of the predetermined range, and 
the target pixel is recognized as being not a noise if 
the following inequality is satisfied. 
Threshold 1 < 

(Value of Pixel of Interest / Average of Reference 

Area) 

< Threshold 2 

If the pixel of interest is regarded as a noise because 
the above inequality is not satisfied, noise may be 
removed by replacing the value of the pixel of interest 
with the average value of the reference area, for 
example. In this case, the area of reference may be 
provided around the center of the pixel of interest for 
areas of great importance, and a large area of reference 
may be provided around the center of the pixel of 
interest for areas of less importance. 

For areas of great importance, a check as to 
whether the pixel of importance is a noise may be made 
with respect to all the RGB signals, and, for areas of 
less importance, the check: may be made with respect to 
only the G signal to which human vision is most 
sensitive. This simplifies the processing. 



The first through fourth embodiments have been 
described above. In these embodiments, a predetermined 
amount of time for the purpose of noise removal may be 
set aside in advance separately with respect to each of 
the areas 1, 2 , and 3 having the respective levels of 
importance as shown in Fig. 11, for example, and the area 
of reference and the processing steps may be determined 
in such a manner as to insure that the noise removal 
will be completed within the predetermined amount of 
time that have been set aside. In this manner, it is 
possible to maintain high image quality in the areas of 
importance perceived by the camera user even though the 
same amount of processing time is spent as when the 
universal processing is applied across the board to the 
entirety of the image. 

Further, the processing of determining the 
number of gray scale levels, the color interpolation 
processing, the sharpness enhancement processing, and 
the noise removal processing that have been described in 
the first through fourth embodiments are not limited to 
the specific examples of these embodiments. As long as 
being applicable to image data that the present 
invention employs , any technology used in the art of 
digital image processing or in the relevant art of image 
processing may be used with the present invention to 



achieve efficient image processing based on the position 
of the fixation point of the user eye. 

Although the present invention has been 
described with reference to specific embodiments, the 
present invention is not limited to these embodiments, 
but various variations and modifications may be made 
without departing from the scope of the invention set 
forth in the claims attached hereto. 
[Advantages of the Invention] 

In the present invention, the number of gray 
scale levels is changed according to the levels of 
importance corresponding to the respective areas of the 
image, and is decreased in the areas of small importance, 
thereby producing an image having varying image quality 
for respective areas to reflect the intension of the 
camera user. By decreasing the number of gray scale 
levels in the areas of small importance, it is possible 
to reduce the memory size and to step up the processing 
speed of image processing that is to be performed at 
later stages. 

In the present invention, further, the color 
interpolation processing of changing color interpolation 
processing for the respective areas of the image, the 
sharpness enhancement processing of changing sharpness 
enhancement processing for the respective areas of the 
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±mage, and/or the noise removal processing of changing 
noise removal processing for the respective areas of 
image are performed according to the determinations by 
the importance computation unit. In this manner, the 
5 color interpolation processing, the sharpness 

enhancement processing, and/or the noise removal 
processing are changed in accordance with the levels of 
importance of the respective image areas , thereby 
producing an image having varying image quality for 
10 respective areas to reflect the intension of the camera 
user. By simplifying the processing in the areas of 
small importance, it is possible to achieve high-speed 
processing . 

In this manner, processing that places an emphasis 
15 on areas of great importance is performed to reflect the 

intension of the camera user, thereby providing a 

digital still camera that can improve image quality in a 

short processing time. 

[Brief Description of the Drawings] 
20 [Fig.l] 

This is a block diagram showing a 
configuration of an embodiment of a digital camera 
according to the present invention. 
[Fig. 2] 

25 This is a drawing showing an embodiment of a 
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configuration of a line-of -sight detection unit. 
[Fig. 3] 

This is a drawing showing the way a display 
screen is divided into a plurality of blocks in order to 
5 detect a point of fixation by the unit of one block. 
[Fig. 4] 

This is a drawing showing the importance of 
the pixel of interest and positional relationship 
between the area of importance and the pixel of interest. 
10 [Fig. 5] 

This is a drawing showing a Gaussian 
distribution function that defines levels of importance. 
[Fig. 6] 

This is a drawing showing the way two areas of 
15 importance are specified in an image. 
[Fig. 7] 

This shows the importance of position (x, y) 
in the image with respect to the first area of 
importance and the second area of importance. 
20 [Fig. 8] 

This is a drawing showing a contour pattern of 
importance in a case where the area of importance is 
defined as a small ellipse region of the image. 
[Fig. 9] 

25 This is a drawing showing an example of 
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importance that is quantized into five levels. 
[Fig. 10] 

This is a block diagram showing a functional 
block configuration of an image processing unit. 
5 [Fig. 11] 

This is a drawing showing quantized importance 
levels that are assigned to respective areas of an image 
with reference to an example in which the area of 
importance is an ellipse shape. 
10 [Fig. 12] 

This is an example of blocks into which an 
image is divided horizontally and vertically. 
[Fig. 13] 

This is an example of blocks having different 
15 shapes into which an image is divided. 
[Fig. 14] 

This is a block diagram showing an example of 
a hardware configuration including an image output 
apparatus . 
20 [Fig. 15] 

This is a flowchart showing a first embodiment 
of a process of making image quality vary depending on 
levels of importance . 
[Fig. 16] 

25 This is a flowchart showing a second 
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embodiment of a process of making image quality vary 
depending on levels of importance . 
[Fig. 17] 

This is a drawing showing a configuration of a 
5 color filter in the case of a camera device being a 
primary' color array CCD. 
[Fig. 18] 

This is a drawing showing the pixel of 
interest and four neighboring pixels for the purpose of 
10 color interpolation processing. 
[Fig. 19] 

This is a drawing showing an example of a wide 
area of interpolation reference that corresponds to the 
case of great importance. 
15 [Fig. 20] 

This is a flowchart showing a third embodiment 
of a process of making image quality vary depending on 
levels of importance. 
[Fig. 21] 

20 This is a drawing showing the pixel of 

interest and neighboring pixels for the purpose of 
sharpness enhancement processing. 
[Fig. 22] 

This is a drawing showing a non- linear 
2 5 transformation applied to Laplacian operation. 
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[F±g.23] 

This is a flowchart showing a fourth 
embodiment of a process of making image quality vary 
depending on levels of importance. 
5 [Description of Reference Numerals] 

1 camera lens 

2 solid camera device (CCD) 

3 A/D conversion unit 

4 camera unit 

10 5 image processing unit 

6 image compression unit 

7 image processing/compression unit 

8 memory unit 

9 PC display 

15 10 line-of -sight detection unit 

11 line-of -sight data storage unit 

12 important area identifying unit 

13 importance computation unit 

14 importance storage unit 

20 15 image quality determining unit 



